{
 "cells": [
  {
   "cell_type": "raw",
   "metadata": {
    "raw_mimetype": "text/restructuredtext"
   },
   "source": [
    ".. _nb_cmaes:"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## CMA-ES\n",
    "\n",
    "\n",
    "**Disclaimer:** We make use of the implementation available at [PyPi](https://pypi.org/project/cma/) <cite data-cite=\"pycma\"></cite> published by the author Nikolaus Hansen under the BSD license. \n",
    "\n",
    "\n",
    "CMA-ES which was proposed in <cite data-cite=\"cmaes\"></cite>. Moreover, a comparing review can be found in <cite data-cite=\"cmaes-review\"></cite>. \n",
    "CMA-ES stands for covariance matrix adaptation evolution strategy. Evolution strategies (ES) are stochastic, derivative-free methods for numerical optimization of non-linear or non-convex continuous optimization problems. They belong to the class of evolutionary algorithms and evolutionary computation. An evolutionary algorithm is broadly based on the principle of biological evolution, namely the repeated interplay of variation (via recombination and mutation) and selection: in each generation (iteration) new individuals (candidate solutions) are generated by variation, usually in a stochastic way, of the current parental individuals. Then, some individuals are selected to become the parents in the next generation based on their fitness or objective function value \n",
    "$f(x)$. Like this, over the generation sequence, individuals with better and better $f$-values are generated.\n",
    "(excerpt from [Wikipedia](https://en.wikipedia.org/wiki/CMA-ES))."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    " "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Example"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "code": "algorithms/usage_cmaes.py"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Best solution found: \n",
      "X = [0.49999997 0.49999994 0.5        0.49999996 0.5        0.5\n",
      " 0.49999996 0.5        0.50000003 0.50000004]\n",
      "F = [0.49999997 0.49999994 0.5        0.49999996 0.5        0.5\n",
      " 0.49999996 0.5        0.50000003 0.50000004]\n",
      "CV= [0.]\n"
     ]
    }
   ],
   "source": [
    "from pymoo.algorithms.so_cmaes import CMAES\n",
    "from pymoo.factory import get_problem\n",
    "from pymoo.optimize import minimize\n",
    "\n",
    "problem = get_problem(\"sphere\")\n",
    "\n",
    "algorithm = CMAES()\n",
    "\n",
    "res = minimize(problem,\n",
    "               algorithm,\n",
    "               seed=1,\n",
    "               verbose=False)\n",
    "\n",
    "print(f\"Best solution found: \\nX = {res.X}\\nF = {res.X}\\nCV= {res.CV}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "CMA-ES already has several stopping criteria implemented. However, as for other algorithms, the number of iterations or function evaluations can be directly passed to `minimize`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "==============================================================================\n",
      "n_gen |  n_eval |     fopt     |    sigma     | min std  | max std  |   axis  \n",
      "==============================================================================\n",
      "    1 |      60 |  0.413583651 |  0.500000000 |  0.50000 |  0.50002 |  1.00005\n",
      "    2 |      70 |  0.413583651 |  0.443532600 |  0.42922 |  0.44527 |  1.10591\n",
      "    3 |      80 |  0.413583651 |  0.406582613 |  0.37335 |  0.41067 |  1.17817\n",
      "    4 |      90 |  0.413583651 |  0.384570210 |  0.34673 |  0.39056 |  1.20631\n",
      "    5 |     100 |  0.413583651 |  0.374835420 |  0.33287 |  0.41146 |  1.32408\n",
      "    6 |     110 |  0.413583651 |  0.365537593 |  0.31713 |  0.40771 |  1.39290\n",
      "    7 |     120 |  0.413583651 |  0.394854291 |  0.34417 |  0.42415 |  1.46804\n",
      "    8 |     130 |  0.413583651 |  0.416364366 |  0.35289 |  0.44742 |  1.55281\n",
      "    9 |     140 |  0.413583651 |  0.389834619 |  0.32470 |  0.42030 |  1.65435\n",
      "   10 |     150 |  0.413583651 |  0.373800540 |  0.30839 |  0.39783 |  1.65274\n",
      "Best solution found: \n",
      "X = [0.36998631 0.38142292 0.55648563 0.73116207 0.84201565 0.31814345\n",
      " 0.41297808 0.4460369  0.7083321  0.84937005]\n",
      "F = [0.41358365]\n"
     ]
    }
   ],
   "source": [
    "res = minimize(problem,\n",
    "               algorithm,\n",
    "               ('n_iter', 10),\n",
    "               seed=1,\n",
    "               verbose=True)\n",
    "\n",
    "print(\"Best solution found: \\nX = %s\\nF = %s\" % (res.X, res.F))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "==============================================================================\n",
      "n_gen |  n_eval |     fopt     |    sigma     | min std  | max std  |   axis  \n",
      "==============================================================================\n",
      "    1 |      60 |  0.230829022 |  0.500000000 |  0.50000 |  0.50002 |  1.00005\n",
      "Best solution found: \n",
      "X = [0.39410806 0.31538908 0.76782696 0.38441356 0.42885057 0.43778444\n",
      " 0.60280162 0.65186544 0.35727118 0.30633172]\n",
      "F = [0.23082902]\n"
     ]
    }
   ],
   "source": [
    "res = minimize(problem,\n",
    "               algorithm,\n",
    "               ('n_evals', 50),\n",
    "               seed=1,\n",
    "               verbose=True)\n",
    "\n",
    "print(\"Best solution found: \\nX = %s\\nF = %s\" % (res.X, res.F))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Also, easily restarts can be used which are known to work very well on multi-modal functions. For instance, `Rastrigin` can be solved rather quickly by:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Best solution found: \n",
      "X = [ 4.50346187e-05 -7.31912594e-05]\n",
      "F = [1.46514089e-06]\n"
     ]
    }
   ],
   "source": [
    "problem = get_problem(\"rastrigin\")\n",
    "\n",
    "algorithm = CMAES(restarts=10, restart_from_best=True)\n",
    "\n",
    "res = minimize(problem,\n",
    "               algorithm,\n",
    "               ('n_evals', 2500),\n",
    "               seed=1,\n",
    "               verbose=False)\n",
    "\n",
    "print(\"Best solution found: \\nX = %s\\nF = %s\" % (res.X, res.F))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Our framework internally calls the `cma.fmin2` function. All parameters which can be used there either as a keyword argument or an option can also be passed to the `CMAES` constructor as well.\n",
    "An example with a few selected `cma.fmin2` parameters is shown below:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Best solution found: \n",
      "X = [-1.66760792e-07  7.76127578e-08]\n",
      "F = [6.71107614e-12]\n"
     ]
    }
   ],
   "source": [
    "import numpy as np\n",
    "from pymoo.util.normalization import denormalize\n",
    "\n",
    "# define an intitial point for the search\n",
    "np.random.seed(1)\n",
    "x0 = denormalize(np.random.random(problem.n_var), problem.xl, problem.xu)\n",
    "\n",
    "algorithm = CMAES(x0=x0,\n",
    "                 sigma=0.5,\n",
    "                 restarts=2,\n",
    "                 maxfevals=np.inf,\n",
    "                 tolfun=1e-6,\n",
    "                 tolx=1e-6,\n",
    "                 restart_from_best=True,\n",
    "                 bipop=True)\n",
    "\n",
    "res = minimize(problem,\n",
    "               algorithm,\n",
    "               seed=1,\n",
    "               verbose=False)\n",
    "\n",
    "print(\"Best solution found: \\nX = %s\\nF = %s\" % (res.X, res.F))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "For more details about hyperparameters we refer to the software documentation of the `fmin2` in CMA-ES which can be found [here](http://cma.gforge.inria.fr/apidocs-pycma/cma.evolution_strategy.html#fmin2).\n",
    "A quick explanation of possible parameters is also provided in the API documentation below."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "raw_mimetype": "text/restructuredtext"
   },
   "source": [
    "### API"
   ]
  },
  {
   "cell_type": "raw",
   "metadata": {
    "raw_mimetype": "text/restructuredtext"
   },
   "source": [
    ".. autoclass:: pymoo.algorithms.so_cmaes.CMAES\n",
    "    :noindex:"
   ]
  }
 ],
 "metadata": {
  "celltoolbar": "Raw Cell Format",
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.4"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}
